1. Field of the Invention
The invention is directed to multimedia signal processing and, more particularly, to video encoding and decoding.
2. Description of the Related Art
Multimedia signal processing systems, such as video encoders, may encode multimedia data using encoding methods based on international standards such as MPEG-x and H.26x standards. Such encoding methods generally are directed towards compressing the multimedia data for transmission and/or storage. Compression is broadly the process of removing redundancy from the data.
A video signal may be described in terms of a sequence of pictures, which include frames (an entire picture), or fields (e.g., an interlaced video signal comprises fields of alternating odd or even lines of a picture). As used herein, the term “frame” refers to a picture, a frame or a field. Video encoding methods compress video signals by using lossless or lossy compression algorithms to compress each frame. Intra-frame coding (herein referred to as intra-coding) refers to encoding a frame using that frame. Inter-frame coding (herein referred to as inter-coding) refers to encoding a frame based on other, “reference,” frames. For example, video signals often exhibit spatial redundancy in which portions of video frame samples near each other in the same frame have at least portions that match or at least approximately match each other.
Multimedia processors, such as video encoders, may encode a frame by partitioning it into blocks or “macroblocks” of, for example, 16×16 pixels. The encoder may further partition each macroblock into subblocks. Each subblock may further comprise additional subblocks. For example, subblocks of a macroblock may include 16×8 and 8×16 subblocks. Subblocks of the 8×16 subblocks may include 8×8 subblocks, and so forth. As used herein, the term “block” refers to either a macroblock or a subblock.
One compression technology based on developing industry standards is commonly referred to as “H.264” video compression. The H.264 technology defines the syntax of an encoded video bitstream together with the method of decoding this bitstream. In one aspect of an H.264 encoding process, an input video frame is presented for encoding. The frame is processed in units of macroblocks corresponding to the original image. Each macroblock can be encoded in intra or inter mode. A predicted macroblock is formed based on portions of an already reconstructed frame or already reconstructed neighboring blocks in the same frame known as causal neighbors. In intra mode, a macroblock is formed from causal samples in the current frame that have been previously encoded, decoded, and reconstructed. Multimedia samples of one or more causal neighboring macroblocks are subtracted from the current macroblock being encoded to produce a residual or difference macroblock, D. This residual block D is transformed using a block transform and quantized to produce X, a set of quantized transform coefficients. These transform coefficients are re-ordered and entropy encoded. The entropy encoded coefficients, together with other information for decoding the macroblock, become part of a compressed bitstream that is transmitted to a receiving device.
Unfortunately, during the transmission process, errors in one or more macroblocks may be introduced. For example, one or more degrading transmission effects, such as signal fading, may cause the loss of data in one or more macroblocks. As a result, error concealment has become critical when delivering multimedia content over error prone networks such as wireless channels. Error concealment schemes make use of the spatial and temporal correlation that exists in the video signal. When errors are encountered, recovery may occur during entropy decoding. For example, when packet errors are encountered, all or part of the data pertaining to one or more macroblocks or video slices (groups of usually neighboring macroblocks) could be lost. When the video data of a slice is lost, resynchronization of decoding can take place at the next slice, and missing blocks of the lost slice can be concealed using spatial concealment.
Since the decoded data available to a decoder device includes the causal neighbors that have already been decoded and reconstructed, spatial concealment typically uses causal neighbors to conceal the missing blocks. One reason for using the causal neighbors to conceal the lost blocks is that out-of-order reconstruction of the next slice followed by concealment of the lost section of the current slice can be very inefficient, especially when using a highly pipelined video hardware decoder core. The non-causal neighbors could offer valuable information for improved spatial concealment. What is needed is an efficient method for providing out of order reconstruction of non-causal neighboring multimedia samples.